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The physical effort required to seek out and extract a resource is an important 
consideration for a foraging animal. A second consideration is the variability or risk 
associated with resource delivery. An intriguing observation from ethological studies is 
that animals shift their preference from stable to variable food sources under conditions of 
increased physical effort or falling energetic reserves. Although theoretical models for this 
effect exist, no exploration into its biological basis has been pursued. Recent advances 
in understanding the neural basis of effort- and risk-guided decision making suggest 
that opportunities exist for determining how effort influences risk preference. In this 
review, we describe the intersection between the neural systems involved in effort- and 
risk-guided decision making and outline two mechanisms by which effort-induced changes 
in dopamine release may increase the preference for variable rewards. 
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INTRODUCTION 

Foraging animals must often consider the variability or risk asso- 
ciated with resource delivery. For example, the preference for 
variable as opposed to stable sources of food can change dramat- 
ically based on a number of factors, such as an animal's current 
energetic state. In the case of food delivery, animals can express 
a preference for variable or "risky" food sources when the ener- 
getic demands of foraging are high (Caraco, 1981; Caraco et al., 

1990) . Juncos, for instance, prefer seed-bins with variable over 
fixed reinforcement schedules when ambient temperatures fall 
and metabolic demands rise (Caraco et al., 1990; but see Brito 
E Abreu and Kacelnik, 1999). Similarly, rats consuming high- 
calorie foods exhibit a reduced preference for variable schedules 
of reinforcement relative to groups of rats consuming low-calorie 
foods (Craft et al., 2011). Energetic expenditure due to physical 
effort can also enhance risk preference. For example, a study by 
Kirshenbaum et al. (2003) investigated the interaction between 
effort and risk in rats where effort was presented in the form of 
running in wheels with low (50 g) or high (120 g) spinning resis- 
tances. Risk was measured as the preference for variable or fixed 
schedules of reinforcement. The authors divided rats into high- 
effort and low-effort groups and observed that the high-effort 
group expressed a preference for variable reinforcement relative 
to the low-effort group. One explanation for this effect is the Daily 
Energy Budget (DEB) or Z-score rule (Stephens, 1981; Houston, 

1991) . According to this theory, when animals approach a critical 
energetic state (e.g., starvation), they choose to gamble by invest- 
ing their remaining resources on a variable option that may yield 
a life-saving gain. This rule also states that organisms with ade- 
quate and stable energetic reserves will not make the gamble as 
they are better served by low-risk options that guarantee survival 



(For a thorough and critical review see Kacelnik and Bateson, 
1996). 

To our knowledge, no investigation into the neural basis for the 
influence of energetic state or physical effort on risk preference 
has been performed. Consequently, almost nothing is known 
regarding the neural basis for interactions between effort and 
risk. Opportunities do exist to explore this question given recent 
advances in the study of the neural basis of effort- and risk- 
guided decision making (Reviewed in Walton et al, 2006; Piatt 
and Huettel, 2008; Salamone et al., 2012). These studies have 
identified a range of structures and neuromodulators involved in 
perceiving and setting the preference to exert effort or to pur- 
sue risky options. Surprisingly, the divergent networks involved 
in effort and risk intersect in a restricted set of cortical and 
subcortical structures; the most prominent being the anterior cin- 
gulate cortex (ACC), the basolateral amygdala (BLA), the nucleus 
accumbens (NAC), and the mesolimbic dopaminergic system. 
Given the considerable evidence for the role of these systems in 
setting the preference for both effort and risk, we believe that 
interactions between these structures underlie the capacity of 
physical effort to increase risk preference. To argue this point, we 
first review evidence that the neural processes involved in effort- 
guided (section Experimental Approaches for Studying Effort- 
Guided Behaviors, Cortical and Sub-Cortical Circuits Involved 
in Effort-Guided Behaviors and Dopamine and Effort-Guided 
Behavior) and risk-guided (section Neural Systems Involved in 
Setting The Preference for Risk) decision making intersect in these 
systems, and conclude with the discussion of a proposed neu- 
ral mechanism by which effort increases the preference for risk, 
where risk preference is defined as a preference for variable over 
certain rewards. 
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EXPERIMENTAL APPROACHES FOR STUDYING 
EFFORT-GUIDED BEHAVIORS 

An underlying assumption of most investigations of effort-guided 
decision making is that effort is a cost to be minimized. This is a 
reasonable assumption given that many species balance energetic 
losses from physical effort against food intake in order to max- 
imize net energetic gain (Stephens et al., 2007). Bautista et al. 
(2001) explored this issue in an experiment in which starlings 
choose between either walking (low effort) in search of low-yield 
rewards or flying (high effort) to reach larger rewards. Analysis 
of choice behavior in response to manipulations of reward mag- 
nitude and flight distance indicated that starlings optimized net 
energetic gain by shifting their preference between walking and 
flying. Similarly, food-restricted rats alter their preference for 
high-effort/high-reward and low-effort/low-reward options in 
ways that indicate that they treat effort as a cost (Reviewed in 
Walton et al., 2006; Salamone et al., 2012). In most neurobio- 
logical experiments, effort is manipulated using either lever-press 
behaviors coupled with variable or fixed-ratio schedules of rein- 
forcement (Floresco et al., 2008; Walton et al, 2009) or barrier- 
climbing tasks where animals are required to select between paths 
on a maze that lead to combinations of effort (the presence of 
a 30-40 cm barrier) and reward (e.g., Salamone, 1994; Bardgett 
et al., 2009; Cowen et al., 2012). Importantly, results from a num- 
ber of studies suggest that the observed aversion to effort reported 
in these studies is not due to an aversion to temporal delays pro- 
duced by the time required to lever-press or jump over barriers 
(Rudebeck et al., 2006; Floresco et al, 2008; Walton et al, 2009). 
What follows is a review of the involvement of neural systems in 
effort-guided behaviors, with a focus on those systems that are 
also implicated in processing risk. 

CORTICAL AND SUB-CORTICAL CIRCUITS INVOLVED IN 
EFFORT-GUIDED BEHAVIORS 

Results from studies that have selectively targeted cortical and 
subcortical structures, either through lesions or through phar- 
macological manipulations, suggest the existence of a network 
involved in effort-guided decision making; with the principal 
nodes of this network being localized in the ACC, BLA, and NAC. 
For example, lesions or inactivation of the ACC result in animals 
avoiding previously preferred high-effort/high-reward options on 
the barrier-climbing task (Walton et al, 2003; Rudebeck et al., 
2006; Floresco and Ghods-Sharifi, 2007). Notably, the placement 
of a second barrier on the formerly low-effort arm results in 
lesioned animals returning to the high-reward/high-effort option 
(Walton et al., 2002), suggesting that the observed shift in prefer- 
ence is not a consequence of an inability to climb the barrier or 
in capacity to recall reward values. This effect does not appear 
to extend to prefrontal regions ventral to the ACC as lesions 
of the prelimbic (Walton et al., 2003) and orbitofrontal cortex 
(Rudebeck et al., 2006) do not result in effort-avoidance. Similar 
patterns of effort-avoidance have been observed following dis- 
ruption of subcortical regions such as the NAC and BLA. For 
example, effort avoidance has been reported in rats perform- 
ing lever-press tasks following muscimol/baclofen inactivation 
of the NAC core (Ghods-Sharifi and Floresco, 2010) and on 
barrier-climbing tasks following the infusion of dopaminergic 



antagonists into the NAC (Salamone et al, 1994) or 6-OHDA 
dopamine depletion of the NAC (Mai et al., 2012). Similarly, 
muscimol/baclofen inactivation of the BLA (Ghods-Sharifi et al., 
2009) on a lever-press task and excitotoxic lesions of the BLA in a 
barrier-climbing task (Ostrander et al., 201 1) also produce effort- 
aversion in rats. Taken together, these data suggest the presence 
of a distributed cortical-subcortical network for effort process- 
ing. A schematic of this network along with possible modes of 
interaction is presented in Figure 1. 

Although experiments such as those described above have 
identified individual structures involved in effort-related pro- 
cessing, little is known regarding the specific roles these struc- 
tures play in effort-guided decision making or how interactions 
between structures influence behavior. Important steps in this 
direction have been made through the investigation of the behav- 
ioral consequences of the functional disconnection of structures 
within this "effort network." For example, the functional discon- 
nection of the BLA and ACC through the targeted inactivation 
of the BLA/ACC in contralateral hemispheres (disconnecting 
within-hemisphere communication) results in a degree of effort- 
avoidance that is similar to the degree observed following bilateral 
ACC or BLA lesions (Floresco and Ghods-Sharifi, 2007). In this 
study, lesions were in the most caudal region of the BLA, a region 
that does not receive significant input from the ACC (Sesack 
et al., 1989). This suggests that information could move from the 
BLA to the ACC during effort-guided decision making. Evidence 
for directional flow of information from the BLA to the ACC, 
along with an established role of the BLA in ascribing positive 
and negative affective values to stimuli and outcomes (Baxter 
and Murray, 2002) indicates that information about the effort- 
discounted value of rewards could be transferred from the BLA 
to the ACC during decision making (Floresco and Ghods-Sharifi, 
2007). The ACC, given its role in integrating sensory and motor 
information in the service of action-selection (Shima and Tanji, 
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FIGURE 1 | Schematic of potential functions of neural systems within 
the effort network. Results from studies in rodents and primates suggest 
the involvement of the anterior cingulate cortex (ACC), basolateral amygdala 
(BLA), nucleus accumbens (NAC), and dopamine neurons in the ventral 
tegmental area (VTA) in effort-guided decision making. Captions in each box 
indicate a proposed roles for each structure in effort-guided behavior and 
arrows indicate possible routes of information flow. These routes are based 
largely from the results of disconnection studies and anatomical studies 
performed in rodents. 
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1998; Kennerley et al., 2009), may utilize information about out- 
come value from regions such as the BLA to bias action selection 
toward high-value options (Walton et al., 2006). This interpreta- 
tion, however, must be tempered as it is possible that the ACC 
drives activity in the posterior BLA through indirect routes (e.g., 
through the anterior BLA). Future experiments involving simul- 
taneous recording from the ACC and BLA during effort-guided 
behavior could further elucidate the question of the direction of 
information flow. 

Interactions between the ACC and the NAC also appear to 
influence effort-guided behaviors. For example, the excitotoxic 
disconnection of the ACC and NAC core also results in effort- 
avoidance on a barrier-climbing task (Hauber and Sommer, 
2009). There are multiple routes through which an ACC-NAC 
disconnection could disrupt inter-region communication. For 
example, projections from the ACC to NAC exist (Zahm and 
Brog, 1992; Voorn et al., 2004; Haber et al, 2006), although 
these are relatively weak (Gabbott et al., 2005). A second route 
is through the ventral tegmental area (VTA) as the ACC projects 
to the VTA (Oades and Halliday, 1987) and given that ACC 
stimulation is capable of activating VTA neurons (Gariano and 
Groves, 1988), and the VTA, in turn, projects to the NAC (Oades 
and Halliday, 1987). In contrast to the clear direct and indirect 
projections from the ACC to the NAC, there are few projec- 
tions from the NAC to the ACC (Hoover and Vertes, 2007), 
suggesting that information flows from the ACC to the NAC 
(Hauber and Sommer, 2009). The specific involvement of the 
NAC in motivated approach behavior (Berridge and Robinson, 
2003; Salamone et al., 2007) suggests that the NAC receives ACC 
output related to the chosen action, and utilizes this information 
to trigger or sustain an approach response. 

In agreement with the proposed role of the ACC in action 
selection (Shima and Tanji, 1998), the preceding results sug- 
gests that the ACC integrates information about outcome values 
from regions such as the BLA in order to guide decision mak- 
ing. Support for a role of the ACC in action selection, however, 
is weakened by recent physiological observations in rodents that 
indicate that neural responses in the ACC to actions and to antic- 
ipated effort actually occur after action selection (Cowen et al., 
2012). These observations are in agreement with an alternative 
view of ACC involvement in sustaining goal-directed responses 
across delays (Cowen et al, 2012). For example, Narayanan and 
Laubach (2006) demonstrated that ACC activity is required for 
sustaining a lever-press response in a time-estimation task. This 
conclusion is further supported by observations from primate 
studies that implicate the ACC in maintaining actions across trials 
(Kennerley et al, 2006; Hayden et al., 201 1). 

It must be cautioned that while these observations highlight 
similarities between primates and rodents, such comparisons are 
complicated by notable anatomical and functional differences 
between species (Preuss, 1995; Uylings, 2003; but see Vogt and 
Paxinos, 2012). For example, there is considerable evidence that 
the primate ACC can be divided into motor, cognitive, and affec- 
tive components along the dorso-ventral/posterior-anterior axis 
(Barbas and De Olmos, 1990; Luppino et al., 1991; Bates and 
Goldman-Rakic, 1993; Barbas and Blatt, 1995; Picard and Strick, 
1996), with most studies of effort-driven responses in primates 



targeting dorsal/motor regions (e.g., Kennerley et al., 2009). 
Although anatomical divisions along the anterior-posterior axis 
of the rodent ACC do appear to correspond generally to the divi- 
sions reported in primates (Vogt and Paxinos, 2012), few studies 
of behavioral/functional differences along this anterior-posterior 
axis in the rodent ACC have been performed. Interestingly, dif- 
ferences in motor sensitivity along the dorso-ventral axis of the 
rodent medial prefrontal cortex have been observed, with indi- 
vidual neurons in dorsal regions, including the ACC, exhibiting 
the greatest sensitivity to movement (Cowen and McNaughton, 
2007). 

DOPAMINE AND EFFORT-GUIDED BEHAVIOR 

Dopamine is traditionally viewed as a key modulator of reward- 
driven learning; however, the observation that dopamine mod- 
ulates effort-guided behaviors suggests that this view requires 
elaboration (Wise, 2004; Salamone et al., 2012). For example, 
physical exercise in the form of swimming and running on a 
wheel or a treadmill and has been repeatedly associated with 
increases in dopamine concentration (Reviewed in Meeusen and 
De Meirleir, 1995). Such increases have been observed through- 
out the striatum (Freed and Yamamoto, 1985; Hattori et al., 
1994; Meeusen et al, 1997), and a particularly strong relation- 
ship between dopamine concentration and running speed has 
been reported in the NAC (Freed and Yamamoto, 1985). In addi- 
tion, rats receiving systemic administration of Dl (Bardgett et al., 
2009) and D2 antagonists (Denk et al., 2005; Floresco et al., 
2008; Bardgett et al, 2009; Salamone et al, 2012) shift their 
preference away from high-effort/high-reward options to low- 
effort/low-reward alternatives. Conversely, the application of the 
dopaminergic agonist D-amphetamine biases responses toward 
high-reward/high-effort options (Bardgett et al., 2009; but see 
Floresco et al, 2008). 

The influence of dopamine on effort depends on dopamine's 
effect on specific structures within the effort-network. For 
instance, targeted 6-OHDA depletion of dopaminergic terminals 
or blockade of dopamine receptors within the NAC (Salamone, 
1994; Cousins et al, 1996; Ishiwari et al, 2004) and the ACC 
(Schweimer and Hauber, 2006 but see Walton and Croxson, 
2005) results in effort-avoidance during cost-benefit decision 
making. Furthermore, results from a microdialysis study of stri- 
atal dopamine during a random-ratio reinforcement task indi- 
cates that slow fluctuations in dopamine concentration (minutes) 
reflect an integration of factors such as motivational state (e.g., 
hunger) and the effort required to obtain rewards (Ostlund et al., 
2011). Taken together, these data indicate that dopamine concen- 
tration within the ACC and the NAC influences the willingness 
of animals to exert physical effort in exchange for valued rewards. 
This function relates to recent theoretical proposals in which the 
level of tonic dopamine release modulates response vigor and 
response speed (Niv et al, 2005) as well as the estimation of 
long-term reinforcement rates (Niv et al., 2007; Ostlund et al., 
2011). 

Although there is clear evidence for a role of dopamine release 
in the NAC in effort-guided behaviors, virtually nothing is known 
regarding how this release is regulated. For example, the principal 
source of dopaminergic innervation of the NAC is the VTA, and it 
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is unknown what systems drive VTA activity during effort-guided 
behaviors. The ACC could be involved given its capacity to trig- 
ger burst-firing in dopaminergic neurons (Gariano and Groves, 
1988), and the previously described role of the ACC in sustaining 
motor actions (Narayanan et al., 2006; Cowen et al., 2012) would 
suggest, somewhat indirectly, ACC involvement in sustaining a 
dopaminergic response. An important path for future research is 
the exploration of interactions between systems within the effort 
network such as the ACC and BLA and the VTA. 

Dopamine can modulate the activity of its targets across many 
time scales (Schultz, 2007), and the time scale most relevant for 
effort-guided behaviors has yet to be determined. For example, 
dopamine neurons can express tonic and phasic modes of firing 
activity (Grace, 1991), and these modes may play distinct roles in 
modulating effort-guided behaviors. Tonic activity is associated 
with slow release (seconds) of dopamine while phasic release is 
associated with brief (~300 ms) bursts of firing activity that pro- 
duces large but temporary increases in dopamine concentration 
(Grace, 1991). There is some debate, however, regarding whether 
changes in extracellular dopamine concentration results from 
tonic or phasic release (Floresco et al, 2003; Owesson-White et al., 
2012). Functionally, tonic patterns of firing activity of dopamine 
neurons have been associated with estimating the uncertainty 
of reward delivery in primates (Fiorillo et al, 2003); while pha- 
sic release is implicated in stimulus-driven orienting behaviors 
(Dommett et al, 2005) and associative plasticity (Hollerman and 
Schultz, 1998). Phasic and tonic patterns of release may also coop- 
erate to regulate inter-region communication (Goto and Grace, 
2005). 

Little is known regarding the relative contribution of phasic 
and tonic release to effort-guided behaviors; however, three recent 
studies using fast scan cyclic voltammetry were unable to identify 
a clear effect of effort on phasic responses within the NAC (Day 
et al, 2010; Gan et al, 2010; Wanat et al, 2010). For example, two 
of these studies were unable to identify significant phasic response 
to effort-predictive cues in an instrumental task, but these studies 
did identify clear phasic responses to reward-predictive cues (Gan 
et al, 2010; Wanat et al, 2010). Furthermore, Day et al. (2010) 
reported an unanticipated reduction of phasic NAC dopamine 
release following increased effort. It is also unclear how short- 
duration phasic responses could exert the sustained effect on 
motivational state required to overcome costs such as physical 
effort (Niv et al., 2007). A possible way to accommodate this 
concern would be if phasic responses modulated the learning of 
associations between effort, stimuli, and outcomes during early 
stages of training as such learning could have lasting effects on 
behavior. Future work in this area may yield important results 
given considerable evidence for a role of phasic dopamine in 
learning (Schultz et al., 1997; Waelti et al, 2001). Indeed, such 
investigations may explain the difficult to resolve issue of how 
effort, under the right conditions, can enhance the reinforc- 
ing value of effort-associated cues, actions, and primary rewards 
(Eisenberger, 1992; Zentall, 2010; Johnson and Gallagher, 2011). 

The variable effects of dopamine on effort-guided behav- 
iors and the different timescales at which dopamine influences 
neural activity present significant challenges to researchers hop- 
ing to create a unifying theory of function. These challenges 



are compounded by experimental results that suggest that the 
involvement of dopamine and the NAC in effort-guided behaviors 
depends on the specific form of physical effort being employed. 
For example, 6-OHDA depletion of NAC dopamine significantly 
impairs effort-guided behavior when effort is assessed using 
fixed-ratio schedules; however, the same dopaminergic manip- 
ulation has no effect when effort is in the form of a weighted 
lever (Ishiwari et al, 2004). One interpretation of these different 
results is that NAC dopamine is required for sustaining effort over 
time. This interpretation, however, does not fit well with results 
from T-maze barrier climbing tasks in which the time required for 
expending effort within a trial is quite brief. Instead, dopamin- 
ergic manipulations may be most effective when sequences of 
effortful motor acts must be maintained. For example, the degree 
of motor control required to press a weighted lever a single time 
is probably similar to the control required for a single press of 
an unweighted lever. In contrast, the control required to exe- 
cute a single lever-press is probably considerably less than what 
is required to sustain a repeating sequence of presses or, in the 
case of barrier-climbing, in executing a complex jumping proce- 
dure. This idea may relate to a formulation of decision-related 
costs proposed by Shenhav et al. (2013) in which cost is the 
degree to which control mechanisms must be engaged during a 
behavior. The authors go further to suggest that the dorsal ACC 
allocates control to a given mental or physical operation based 
on the estimated value of the outcome. This value signal used in 
this allocation process is suggested to arrive from regions such as 
the amygdala, insula, and dopaminergic system (Shenhav et al., 
2013). 

NEURAL SYSTEMS INVOLVED IN SETTING THE PREFERENCE 
FOR RISK 

Studies in humans, non-human primates, and rodents have 
identified diverse cortical and subcortical structures involved 
in setting the preference or aversion to risk. In human stud- 
ies, the Iowa Gambling Task has become an established task 
for investigating the neural basis for risk-guided decision mak- 
ing (Bechara et al., 1994). In this task, subjects choose between 
four decks of cards with the decks having different levels of 
probabilistic gains and losses and different long-term returns. 
Risk-preference is measured as a preference for suboptimal high- 
risk/high-reward decks over optimal low-risk alternatives. A pref- 
erence for high-risk options has been observed in patients with 
damage to the ventromedial prefrontal cortex and amygdala 
(Bechara et al, 1994, 1999). Furthermore, BOLD activation of 
the medial frontal gyrus, lateral orbitofrontal cortex, and insula 
increases when subjects choose suboptimal high-risk options 
(Lawrence et al., 2009). 

A criticism of the use of the Iowa Gambling Task for assessing 
risk-preference is that the task places multiple cognitive demands 
upon subjects that are unrelated to risk. This is problematic as a 
preference for risky alternatives or neural responses related to risk 
preference could, in this task, result from a reduced capacity to 
shift attention, to perform behavioral reversals, to process losses, 
or to store items in working memory (Maia and McClelland, 
2004; Dunn et al., 2006). To address this issue, some fMRI studies 
in humans and most studies in non-human primates and rodents 
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use paradigms that more directly assess the preference or aversion 
to variability in reward delivery. These studies have implicated a 
large number of structures in risk-guided decision making such 
as the lateral frontal cortex (Tobler et al., 2009), medial prefrontal 
cortex (St. Onge and Floresco, 2010), insula (Preuschoff et al., 
2008; Rudorf et al., 2012), posterior parietal cortex (Huettel et al., 
2006), ACC (Christopoulos et al, 2009; Kennerley et al„ 2009), 
posterior cingulate cortex (McCoy and Piatt, 2005), VTA (Fiorillo 
et al, 2003), putamen (Preuschoff et al., 2008), and ventral stria- 
tum (Cardinal and Howes, 2005; Preuschoff et al., 2006; Stopper 
etal, 2013). 

Connecting results from human studies to results from studies 
using non-human primates and rodents is a significant challenge 
given the wide range of structures identified within and across 
species. Even so, there are points of intersection. Interestingly, 
many of these points of intersection include elements of the effort 
network, such as the ACC, NAC, BLA, and dopamine (Amiez 
et al, 2006; Behrens et al., 2007; Pais-Vieira et al, 2007; Piatt 
and Huettel, 2008; Preuschoff et al, 2008; Choi and Kim, 2010; 
Fitzgerald et al, 2010; Rivalan et al., 2011; Schultz et al, 2011; 
Zeeb and Winstanley, 2011). The following sections review the 
involvement of these systems in risk-guided behaviors. 

RISK AND THE ANTERIOR CINGULATE CORTEX 

Multiple lines of evidence suggest a role of the primate ACC in 
evaluating risk and in altering risk preference. In non-human pri- 
mates, the spiking activity of subsets of ACC neurons changes 
monotonically with the probability that a cue predicts reward 
(Kennerley et al, 2009). In humans, BOLD activity in the ACC 
increases prior to decisions involving risk (Labudda et al., 2008; 
Weber and Huettel, 2008; Lawrence et al, 2009; Smith et al, 
2009). Furthermore, the combination of BOLD activation of the 
ACC and striatum predicts risk-preference on subsequent trials 
(Christopoulos et al, 2009), suggesting that ACC/striatal activa- 
tion sets a baseline preference for risk. The ACC may specialize in 
monitoring a form of risk known as volatility or "uncertain uncer- 
tainty," a form of uncertainty in which distributions and means of 
outcome delivery are unknown (Rushworth and Behrens, 2008). 
BOLD activity in the ACC scales with the amount of information 
an outcome provides about the underlying outcome distribu- 
tion (Behrens et al., 2007). For a foraging animal, adapting to 
volatility is important as it determines how much weight to place 
on the history of reward delivery, with high volatility indicat- 
ing that more attention should be paid to recent history. Results 
from lesion studies of the primate ACC suggest that ACC plays 
an important role in tracking reward history (Kennerley et al., 
2006) and so ACC responses to volatility would support this 
function. 

When compared to results from primates, results from studies 
in rodents have not produced consistent evidence for a role of the 
ACC in risk-guided behaviors. Although inactivation (St. Onge 
and Floresco, 2010), lesions (Rivalan et al., 2011), or dopaminer- 
gic blockade (St. Onge et al., 201 1 ) of the medial prefrontal cortex 
does alter risk-preference, studies that have specifically targeted 
the rodent ACC have not reported alterations in risk preference in 
two tests of risk preference (St. Onge and Floresco, 2010; Rivalan 
etal, 2011). 



RISK, DOPAMINE, AND THE NUCLEUS ACCUMBENS 

Dopamine plays a central albeit complex role in modulating 
the preference for risk. For example, systemic administration of 
dopamine Dl and D2 antagonists in rats results in the avoidance 
of high-risk/high-reward options in a 2-lever risk-discounting 
task (St. Onge and Floresco, 2009) while amphetamine or Dl 
or D2 agonists increased the preference for high-risk/high- 
reward options in the same task (St. Onge and Floresco, 2009). 
Conversely, a study that combined the risk of reward with the risk 
of punishment (foot shock) reported that systemic administra- 
tion of D2 agonists and amphetamine reduced risk taking for sub- 
optimal large-reward/punishment options (Simon et al., 2011). 
Similarly, another study reported that the systemic application 
of D2 antagonists resulted in the avoidance of high-risk/reward 
options on a rodent version of the Iowa Gambling Task that 
also incorporated punishment in the form of time-outs (Zeeb 
et al, 2009). The specific effects of systemic D2 antagonism can 
be difficult to interpret given that D2-like receptors are located 
on presynaptic dopaminergic afferents and on the postsynaptic 
terminals of their cortical and striatal targets (Palij et al., 1990; 
Timmerman et al., 1990; Santiago and Westerink, 1991). Taken 
together, these results do indicate that dopamine plays a clear but 
complex role in risk-guided behaviors, and the specific role may 
depend on the task, the presence or absence of punishment, and 
the receptor type that is targeted. 

Results from single unit-studies of dopaminergic neurons in 
the primate VTA also suggest a role of dopamine in evaluating 
risk. For example, the activity of dopaminergic neurons follows 
the probability of receiving a future reward with the activity of 
these neurons peaking when the capacity to predict a reward is 
lowest (Fiorillo et al., 2003; Tobler et al, 2005). Similarly, imaging 
studies of the midbrain in humans show increased BOLD activa- 
tion under conditions of outcome uncertainty (Aron et al, 2004). 
The effects of dopaminergic manipulations in humans also sup- 
port a role for dopamine in shaping risk preference. For example, 
patients undergoing dopamine agonist therapy for the treatment 
of Parkinson's disease and restless leg syndrome exhibit a four- 
fold increase in gambling-related problems (Dodd et al., 2005; 
Dang et al., 201 1), while low doses of amphetamine increase self- 
reports of the desire to gamble in problem gamblers (Zack and 
Poulos, 2004). Interestingly, the application of antagonists that 
targeted only D2 receptors increased the perceived reward of a 
slot-machine gambling episode (Zack and Poulos, 2007). Finally, 
changes in striatal and cortical dopamine receptor populations 
and dopamine transporters (DAT) over an organism's develop- 
ment correspond with age-associated shifts in risk preference 
(Wahlstrom etal, 2010). 

As in effort-guided decision making, the influence of 
dopamine on risk preference may be through dopamine's regula- 
tion of the NAC. For example, inactivation or lesions of the NAC 
in rodents reduces the preference for large, uncertain rewards 
(Cardinal and Howes, 2005; Stopper et al, 2013). Furthermore, 
infusion of Dl antagonists into the NAC of rats decreases the 
preference for larger, uncertain rewards on a probability discount- 
ing task. In contrast, infusion of a Dl or D2/D3 agonists into 
the NAC increases risk preference when reward probability is 
high (Norbury et al., 2013; Stopper et al., 2013; however, see 



www.frontiersin.org 



November 2013 | Volume 7 | Article 208 | 5 



Miller etal. 



Intersections between effort and risk 



Mai and Hauber, 2012). Infusion of Dl agonists into the NAC 
may also "optimize" decision making in treated animals as one 
study reported that treated rats were more likely to choose the 
large and uncertain reward when reward probability was high, 
but were more likely to avoid this option when probabilities were 
low (Stopper et al., 2013). Further, background dopamine levels 
appear to peak on trials during which reward delivery is uncer- 
tain (St. Onge et al., 2012a). Similarly, one study reported that 
BOLD responses in the human NAC were largest under condi- 
tions of maximal uncertainty (Preuschoff et al, 2006). In another 
study, NAC BOLD activation increased before subjects made risky 
choices in a task involving decisions between high (stocks) or 
low (bonds) risk investments (Kuhnen and Knutson, 2005). These 
patterns of activation mirror patterns observed in the firing activ- 
ities of dopaminergic neurons of the VTA of non-human primates 
(Fiorillo et al, 2003; Tobler et al., 2005). 

Taken together, these results suggest that increased NAC 
dopamine enhances the tolerance of risk while reductions in NAC 
dopamine reduces risk preference. A similar pattern was observed 
for effort, with enhanced NAC dopamine resulting in an increased 
willingness to work for rewards and reduced dopamine resulting 
in effort avoidance. The following section presents possible mech- 
anisms by which these similar patterns contribute to the influence 
of effort on risk preference. 

RISK AND THE BAS0LATERAL AMYGDALA 

The BLA is also associated with the assessment of risk, although 
fewer studies have targeted the BLA relative to the regions dis- 
cussed previously. For example, results from a study by Ghods- 
Sharifi et al. (2009) suggest that inactivation of the BLA leads 
to risk-averse behavior. In this study, animals chose between a 
lever that delivered a small, certain reward and a lever that deliv- 
ered a large reward with decreasing reinforcement probabilities 
that shifted from certain (reward on 100% of trials) to risky 
(reward on 12.5% of trials). Rats with bilateral inactivation of 
the BLA preferred the low- risk lever when compared to the saline 
control. In the same study, the authors investigated effort dis- 
counting using a ratio schedule of reinforcement and reported 
that BLA lesions also resulted in effort aversion (Ghods-Sharifi 
et al, 2009). Taken together, this suggests a general role for the 
BLA in overcoming costs associated with both effort and risk. 
However, observations from a recent disconnection study per- 
formed by the same group suggest that the role of the BLA and 
its interaction with other structures is nuanced. In this study, the 
authors disconnected the BLA and mPFC and the BLA and the 
NAC in a similar risk-discounting task (St. Onge et al, 2012b). 
The authors observed that BLA-mPFC disconnection resulted in 
increased risk preference while BLA- NAC disconnection resulted 
in risk aversion. The study further determined that it was the 
mPFC to BLA and not the BLA to mPFC connection that was 
involved in enhancing risk-preference. This finding suggests that 
the mPFC input to the BLA could serve to blunt risk-seeking so 
that animals can optimize foraging behavior over time. 

HOW DOES EFFORT ALTER RISK PREFERENCE? 

Increased metabolic demands and expenditure of physical effort 
can enhance the preference for variable rewards (Caraco, 1980, 



1981; Caraco et al., 1990; Kirshenbaum et al., 2003). It has been 
proposed that a shift in preference for variable rewards is an adap- 
tive behavior that biases animals to gamble on potentially large 
and life-sustaining gains (Stephens, 1981). The neural mecha- 
nisms underlying the capacity of effort to alter risk preference 
are unknown. Broadly, there are at least two routes by which 
effort could influence risk. First, the notable intersection between 
the neural systems involved in effort and risk-guided behaviors 
(e.g., the NAC, ACC, and BLA) and the capacity neurons within 
some of these structures to respond to both effort and risk (e.g., 
Kennerley et al., 2009) suggests that activity in these structures 
that results from anticipated or experienced effort could alter and 
consequently bias the processes within these structures involved 
in evaluating risk. Alternatively, it is possible that neural struc- 
tures that are not typically associated with effort or risk could 
become engaged only when environmental contingencies require 
the integration both of these considerations (Burke et al., 2013). 
For example, Burke et al. (2013) observed that significant BOLD 
activation of the frontal pole was only observed when combina- 
tions of effort and risk were being considered. The authors of 
this study did not investigate the neural basis for the capacity of 
effort to alter risk-preference, and, to our knowledge, no direct 
investigation of this question has been performed. It is therefore 
hoped that the following and admittedly speculative proposals 
will motivate future experiments to fill this gap in understanding. 

EFFORT-TRIGGERED INCREASES IN NAC DOPAMINE CONCENTRATION 
ENHANCES RISK PREFERENCE 

Dopamine is an important modulator of both effort- and risk- 
guided behaviors. Here we propose that the release of dopamine 
in the NAC in response to physical effort enhances risk prefer- 
ence. This idea is supported by the following observations. First, 
dopamine concentrations within the NAC increase with physical 
exercise (Freed and Yamamoto, 1985). Furthermore, increases in 
dopamine concentration are associated with increased risk prefer- 
ence in humans (Zack and Poulos, 2004; Dodd et al., 2005; Dang 
et al., 2011) and rodents (St. Onge and Floresco, 2009; Stopper 
et al, 2013). Finally, stimulation of dopamine Dl receptors within 
the NAC enhances risk preference (Stopper et al., 2013; however, 
see Mai and Hauber, 2012). Consequently, we propose that effort- 
induced increases in NAC dopamine concentration drives risk 
preference by enhancing dopamine release in the NAC. 

What remains to be specified is the mechanism by which 
NAC dopamine concentration is regulated during effort-guided 
behaviors. In this regard, the ACC may play a central role 
given abundant evidence for ACC involvement in effort-guided 
behaviors (Reviewed in section Experimental Approaches for 
Studying Effort-Guided Behaviors, Cortical and Sub-Cortical 
Circuits Involved in Effort-Guided Behaviors and Dopamine and 
Effort-Guided Behavior and see Figure 1). Furthermore, activa- 
tion of the ACC is capable of driving the responses of dopaminer- 
gic neurons in the VTA (Gariano and Groves, 1988), the principal 
source of NAC dopamine. Activity within the dorsomedial pre- 
frontal cortex, including the ACC, is also capable of sustaining 
motor activity during delay intervals (Narayanan and Laubach, 
2006; Narayanan et al., 2006) suggesting that ACC neurons could 
sustain dopaminergic responses and enhance dopaminergic tone 
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in the NAC during effortful behaviors. The increase in NAC 
dopaminergic tone could, as a secondary effect, alter risk-guided 
behaviors. 

The capacity of the ACC to sustain responses over delays may 
be particularly important for maintaining effortful behaviors as 
such behaviors often require maintaining motor responses over 
seconds and minutes. Consequently, understanding the timescale 
of dopamine release in the NAC during effortful behavior may be 
critical for understanding its role in modulating risk-preference. 
As discussed in section Dopamine and Effort-Guided Behavior, 
dopamine release can occur on multiple timescales, and the 
timescale most relevant for effort- and risk-guided behaviors has 
yet to be determined, although evidence is building that phasic 
patterns of dopamine release in the NAC are not critical for effort- 
guided behaviors (Can et al., 2010; Wanat et al., 2010). Instead, 
gradual shifts in dopamine release may modulate effort-guided 
decision making, facilitate the maintenance of effortful responses, 
and sustain a preference for risk. Increases in tonic firing of VTA 
neurons correlates with anticipated risk (Fiorillo et al., 2003), 
suggesting that effort-induced increases in dopamine may alter 
risk-preference. 

Interestingly, a recent study has identified within-trial 
increases in striatal dopamine concentration that ramps up as 
rats approach goal locations on a T-maze (Howe et al., 2013). 
These increases appear to be independent of phasic dopamine 
release and occur at a faster time scale than typically reported 
for changes in background dopamine concentration. The ACC 
could potentially drive this ramping response given the capacity 
of medial prefrontal neurons to sustain firing activity in afferent 
targets (Narayanan and Laubach, 2006; Narayanan et al., 2006). 
As suggested by Howe et al. (2013), the ramping response may be 
involved in setting the ongoing level of motivation. Determining 
the level of motivation could require the integration of factors 
such as effort, risk, and expected reward. 

EFFORT-TRIGGERED INCREASES IN DOPAMINE ENHANCES LEARNING 
UNDER CONDITIONS OF UNCERTAINTY 

The preceding account suggests that effort-induced changes 
in NAC dopamine enhance risk-preference and provides the 
motivational signal required for animals to sustain effortful 
behaviors. An alternative mechanism by which effort could 
influence risk is through dopamine's impact on stimulus- 
reward learning. Dopamine has an established role in facilitat- 
ing such learning (Schultz et al, 1997; Day et al., 2007; Flagel 
et al, 2011; Steinberg et al., 2013), and both tonic and pha- 
sic patterns of dopamine release appear to enhance plasticity 
at hippocampal-NAC synapses (Goto and Grace, 2005). It is 
therefore possible that increases in NAC dopamine from both 
effort (Freed and Yamamoto, 1985) and risk (Fiorillo et al., 
2003; St. Onge et al., 2012a) could produce an additive effect 
on plasticity, and, as a result, an additive effect on stimulus- 
reward learning under conditions that combine risk and effort. 
Anatomical connectivity between the VTA and the amygdala 
(Oades and Halliday, 1987) and the capacity of dopamine to 
gate plasticity within the amygdala (Bissiere et al, 2003) sug- 
gest that enhancements in plasticity could extend to other limbic 
structures. 



An interesting feature of this idea is that it may account for the 
paradoxical capacity of effort to enhance the reinforcing prop- 
erties of cues and primary rewards. Pigeons, for example, prefer 
colored lights, spatial locations, and primary rewards associated 
with high effort (Clement et al., 2000; Friedrich and Zentall, 
2004; Singer et al., 2007). Similar observations have been reported 
in starlings (Kacelnik and Marsh, 2002), mice (Johnson and 
Gallagher, 2011), and humans (Zentall, 2010). Indeed, mice can 
express a preference for low- over high-calorie foods when low- 
calorie foods are paired with high effort (Johnson and Gallagher, 
2011). If effort-triggered dopamine release enhances stimulus- 
reward learning in, for example, the amygdala (Bissiere et al., 
2003), or striatum (Day et al, 2007; Cerovic et al., 2013), then 
such learning could result in a preference for rewards and reward- 
predictive cues learned under conditions of effort. 

CONCLUSION 

The recent trend of using ethologically motivated experimen- 
tal design to connect field observations with observations from 
neuroscience (Glimcher, 2002; Bateson, 2003; Stephens, 2008; 
McNamara and Houston, 2009) has begun to produce interest- 
ing results (e.g., Pearson et al., 2009; Hayden et al., 2011; Kolling 
et al., 2012). One important insight from foraging research is 
that animals are particularly attentive to the variability of reward 
delivery, even when average amounts of rewards are held con- 
stant (Stephens and Krebs, 1987; Kacelnik and Bateson, 1996). 
A second interesting observation is that the preference for such 
variability can be modulated by energetic state (Caraco, 1981) and 
physical effort (Kirshenbaum et al., 2003). Although there is com- 
pelling behavioral evidence for interactions between effort and 
risk, neuroscientific investigations of these interactions have not 
been performed. Given the surprising degree of overlap between 
networks involved in effort- and risk-guided behaviors, there is 
good reason to believe that the neural systems involved in eval- 
uating, processing, and setting the preference for effort and risk 
interact. Consequently, it is hoped that this review will stimulate 
interest in the risk-effort connection and encourage new experi- 
ments that investigate the nuanced role that physical effort plays 
in foraging and decision making. 
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